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ABSTRACT 



This report presents results from four 1999 Urban Systemic 
Initiative (USI) school district surveys. The Survey of Enacted Curriculum is 
the study component of a grant from the National Science Foundation to 
examine how reform works in USI districts. The study explores the impact of 
USI programs on student achievement and the learning infrastructure in urban 
school districts and will develop an inferential causal model linking USI 
drivers and other key elements. This survey analyzes urban school mathematics 
and science practices, focusing on enacted curriculum contents and teaching 
practices. It provides a means of validating reform changes in the four sites 
by analyzing responses from intervention and control group teachers. In each 
site, 80 teachers were selected from 20 elementary and middle schools. The 
initial report on enacted curriculum in USI sites demonstrates that: the 
survey approach tried at the four USI sites can be used to analyze curriculum 
and teaching in classrooms; the analysis can be used across different 
classes, schools, and districts; and a purposeful sample of schools and 
teachers can be used to compare curriculum and instruction in schools that 
have high implementation of systemic reform with USI schools that have less 
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ABSTRACT 



Survey Results of Urban School Classroom Practices 
in Mathematics and Science : 1999 Report 

Using the Survey of Enacted Curriculum 
Conducted during Four USI Site Visits 



The Survey of Enacted Curriculum (SEC) is being conducted as a study component of a three year 
grant from the National Science Foundation (NSF), entitled "How Reform Works: An Evaluative 
Study of NSF's Urban Systemic Initiatives (USI).” The evaluative study explores the impact of USI 
programs on student achievement and the learning infrastructure in urban school districts, and will 
develop an inferential causal model linking the Systemic Initiatives drivers and other key elements. 

Within the context of the USI evaluative study, the survey provides an initial analysis of urban 
school classroom practices in mathematics and science, focusing on enacted curriculum contents and 
teaching practices. The survey has been designed to provide a means of validating reform changes in 
the four USI sites by analyzing responses from two groups of mathematics and science teachers - 
implementation group and control group. In each urban site, 80 teachers were selected from 20 
schools, with 10 schools each at elementary and middle grade levels. 

This initial report on enacted curriculum in USI sites provides two very important kinds of results. 
First, we demonstrate that the survey approach tried with four USI sites in 1999 can be used to 
analyze curriculum and teaching in classrooms, and that the analysis can be used across different 
classes, schools, and districts. Second, the report demonstrates that a purposive sample of schools 
and teachers can be used to compare curriculum and instruction in schools with high implementation 
of systemic reform through USI with schools that have less implementation. 
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CHAPTER I 



INTRODUCTION 



ERIC 



1.1 The Survey of Enacted Curriculum and the Evaluative Study of 
NSF’s Urban Systemic Initiatives 

The Survey of Enacted Curriculum (SEC) is being conducted as a study component 
of a three year grant from the National Science Foundation (NSF), entitled "How 
Reform Works: An Evaluative Study of NSF's Urban Systemic Initiatives (USI).” 

The purpose of the study is to determine the impact of the USI program on student 
achievement and the learning infrastructure in urban school districts. The study is 
focusing on development of an inferential causal model that relates the systemic 
initiative drivers and other key elements to the outcomes observed. 

As part of the study activities, the evaluative study team conducts four site visits each 
year to meet and interview superintendents, principals, teachers, and USI and district 
staff members. During two day site visits to the Baltimore, Dallas, Detroit, and 
Phoenix school districts between April and May 1999, the study team also met with a 
group of teachers to conduct the survey. This monograph presents the survey 
summary and findings relevant to the first year of our evaluative study. The survey 
will be continued in the second and third years of our study period for four selected 
USI sites each year. 

Within the context of the USI evaluation study, there are four main objectives for 
conducting the Survey: 

• Collect data from a sample of teachers in selected urban sites and schools in 
order to analyze classroom practices and curriculum in science and math that will 
help to confirm and validate change in practice related to USIs. 

• Analyze actual classroom teaching practices compared to the Key Indicator Data 
System (KIDS). 

• Analyze current enacted classroom curriculum and instruction in relation to 
expectations for practice as outlined in state or district content standards, science 
and math reform initiatives, and assessments. 

• Develop survey and data analysis tools that provide reliable, valid data for 
measuring and reporting curriculum and instructional practices, as well as teacher 
preparation and background. 



1 




The SEC provides a practical research tool for collecting consistent data on 
mathematics and science instructional practices and curriculum, based on teacher 
reports on what is taught in the classroom. The SEC offers an objective approach to 
analyzing current classroom practices in relation to state, national, and local content 
standards and the goals of systemic initiatives. The survey results provide a means of 
validating described changes in practice in urban systemic sites by analyzing 
classroom practice data from a sample of schools in selected sites. The survey 
instrumentation and method of data reporting will also provide a model for 
subsequent studies and indicators of reform in Urban Systemic Initiatives. 



Urban Systemic Initiative 
21 School Districts 

Cohort 93 

Baltimore 
Chicago 
Dallas 
Detroit 
El Paso 
Miami-Dade 
New York 
Phoenix 

Cohort 94 

Cleveland 
Columbus 
Fresno 
Los Angeles 
Memphis 
New Orleans 
Philadelphia 

Cohort 95 

Milwaukee 
St. Louis 
San Antonio 
San Diego 

Cohort 97 

Atlanta 

Jacksonville 
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1.2 Background of Urban Systemic 
Initiatives and the Evaluative Study 

In 1994, NSF launched the USI program, applying lessons learned from its initial 
State Systemic Initiative (SSI) program to the problems of inner city school systems. 
The USI program was offered to the 25 cities with the largest number of K-12 
students living in poverty. Four cohorts of cities (21 cities in all) signed cooperative 
agreements with NSF for a five-year concerted system-wide effort to promote 
standards-based reform in mathematics, science, and technology (MST). The NSF 
investment was meant to be a catalyst for large-scale educational change affecting 
standards, curriculum, assessment, professional development, partnerships, and 
convergence of intellectual and fiscal resources, with constant attention to improving 
student achievement. 



Over the course of its systemic initiative programs, NSF has developed a theoretical 
structure for systemic reform that is based on six “drivers”; four process drivers and 
two student outcome drivers, as well as a number of cross-cutting issues such as 
equity, quality, scaling up, coordination and organization. NSF is committed to 
measuring impact within its systemic initiatives in the following three areas: the 
implementation of standards-based curriculum, enhanced professional development 
for teachers of mathematics and science, and significant student achievement. 

With a focus on the six drivers outlined by NSF, our evaluative study will pursue 
evidence of program effectiveness that profoundly affected and sustained systemic 
changes in 21 USI cities. We seek to identify interrelationships among the four 
process drivers (Drivers 1 to 4) and two outcome drivers (Drivers 5 and 6) based on 
compiled/surveyed Key Indicator Data from the current USI sites. 

Our study is focusing on two major hypotheses and one investigative question: 



Hypothesis I: 



Hypothesis II: 
Question I: 



A well-implemented USI program has significant and sustainable 
positive impact on the infrastructure that supports student success 
(opportunity to learn) including curriculum, teacher skills, and 
resources in urban school districts. 

A well-implemented USI program leads to significantly improved 
student outcomes in MST. 

Which SI drivers/elements/cross-cutting variables are most critical to 
achieving sustainable reform and improved student outcomes, and 
how do these variables interact with each other? 
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Six Drivers for 
Educational System Reform 

1 . Implementation of comprehensive, standards-based curricula as represented in 
instructional practice, including student assessment, in every classroom, 
laboratory, and other learning experiences provided through the system and its 
partners. 

2. Development of a coherent, consistent set of policies that supports: provision of 
high quality mathematics and science education for each student; excellent 
preparation, continuing education, and support for each mathematics and science 
teacher (including all elementary teachers); and administrative support for all 
persons who work to dramatically improve achievement among all students 
served by the system. 

3. Convergence of the usage of all resources that are designed for or that reasonably 
could be used to support science and mathematics education-fiscal, intellectual, 
material, curricular, and extra-curricular— into a focused and unitary program to 
constantly upgrade, renew, and improve the educational program in mathematics 
and science for all students. 

4. Broad-based support from parents, policymakers, institutions of higher education, 
business and industry, foundations, and other segments of the community for the 
goals and collective value of the program, based on rich presentations of the 
ideas behind the program, the evidence gathered about its successes and its 
failures, and critical discussions of its efforts. 

5. Accumulation of a broad and deep array of evidence that the program is 
enhancing student achievement, through a set of indices that might include 
achievement test scores, higher level courses passed, College admission rates, 
college majors, Advanced Placement Tests taken, portfolio assessment, and 
ratings from summer employers, and that demonstrate that students are generally 
achieving at a significantly higher level in science and mathematics. 

6. Improvement in the achievement of all students, including those historically 
underserved. 
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CHAPTER II 



METHODOLOGY AND INSTRUMENTATION 



11.1 Background 

Under the grant for the USI Evaluative Study, Systemic Research, Inc. contracted 
with the Council of Chief State School Officers (CCSSO) to use the Survey of 
Enacted Curriculum (SEC) as a study component, beginning in Spring 1999. The 
SEC instruments, analyses, and reporting methods were designed and field-tested by 
CCSSO in collaboration with researchers at the National Institute for Science 
Education (NISE) at the University of Wisconsin-Madison. The design and 
development work began in 1995 as part of a multi-state collaborative project, and 
the current survey design was completed under support from the National Science 
Foundation, Education and Human Resources Directorate, Division of Research, 
Evaluation and Communication. A 1998 grant to CCSSO is also supporting a 
concurrent study of state reform initiatives using the SEC design. 



11.2 Sampling and Data Collection 

In each urban site, we purposely selected 20 schools, with 10 schools each at 
elementary and middle grade levels. Five high reform, or Implementation, schools 
per level (based on degree or time in urban systemic implementation) were matched 
with five Comparison schools (schools not in the first phase of the urban systemic 
initiative, and thus with less implementation of reform). We studied elementary and 
middle schools in each site because the focus of reform implementation in the 
majority of USIs is focused at these levels, rather than in high schools. 

The plan was to standardize the data collection in schools by selecting four teachers 
per level. Two completed the teacher survey for science education, and other two 
completed the survey for mathematics education. In each district, the teachers were 
selected at grade 4 or 5 and grade 7 or 8. Thus, the goal was a total of 40 teachers 
(20 mathematics and 20 science teachers) surveyed per grade level, or a total of 80 
teachers per district as summarized in Table 1. The survey forms are prepared in four 
categories: two subjects (mathematics and science) per two grade levels (elementary 
and middle). 




5 



15 



Table 1. Survey Sample Size for a USI School District 



School Level 


High Reform Schools 
(Teachers) 


Comparison Schools 
(Teachers) 


Total 


Elementary 
grade 4 or 5 


5 schools (10 Math & 
10 Science Teachers) 


5 schools (10 Math & 
10 Science Teachers) 


10 schools (20 Math & 
20 Science Teachers) 


Middle 
grade 7 or 8 


5 schools (10 Math & 
10 Science Teachers) 


5 schools (10 Math & 
10 Science Teachers) 


10 schools (20 Math & 
20 Science Teachers) 


Total 


10 schools (20 Math & 
20 Science Teachers) 


10 schools (20 Math & 
20 Science Teachers) 


20 schools (40 Math & 
40 Science Teachers) 



11.3 Survey Procedures 

As previously mentioned, the survey was conducted during each site visit. Table 2 
briefly illustrates a sample site survey procedure. 



Table 2. A Site Survey Procedure Sample 



Step 


Procedure 


Target Date 


1 


Select 4 USI Sites for visits; Brief sites on survey/ school selection 


Jan. 30 


2 


Site submits list of “high reform” and comparison schools 


By Feb. 28 


3 


SR and CCSSO staff review schools and data; contacts sites if needed 


Feb. 


4 


Site contact gains cooperation of schools and provides study information 
School contact names given to staff 


Feb. 


5 


Superintendent or PD sends a letter to participating schools 
Draft tetter and info, on teacher selection provided by staff 


By March 1 


6 


Briefing for school contacts first day of site visit 


April 


7 


Surveys administered during site visit week or distributed with return date 


April 


8 


CCSSO receives surveys, togs in, and follow-up 


April 


9 


Completed surveys/forms transmitted to NSF/UW-Madison for 
scanning/input to data file 


By May 1 5 


10 


Data analysis program and data file for each site provided to SR 


By July 1 



Before each site visit, CCSSO sent out an initial memorandum explaining the 
purpose and methodology of the survey. The USI Project Director selected 20 
schools based on the criteria described in the memo. During the site visit, the 
evaluative study team met a representative teacher from each of the 20 schools to 
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deliver the survey package and instructions. After the site visit, each teacher 
completed a survey and returned the completed forms in a self-addressed, prepaid 
envelope to the University of Wisconsin for data processing and analysis. 

11.4 Survey Instrument Structure 

The survey was conducted using four different instruments to accommodate two 
grade levels and the two subject matters involved. However, each instrument follows 
the general structure as shown in Table 3. 



Table 3. General Structure of SEC Instrument for Middle School Mathematics 



Section 


Questions (number of questions & type) 


1. General School 

and Class Descriptions 


• School Description (2Q- multiple choice) 

• Class Description (1 IQ- multiple choice) 

• Most recent unit of mathematics instruction & instructional time distribution 
(10Q- percentage of time) 


II. Subject Content - 

Expectations for Students 
in Mathematics 


♦ Number sense/ Properties/ Relationships (1 7Q) 

♦ Measurement (13Q) 

♦ Data analysis (15Q) 

♦ Algebra concepts (22Q) 

♦ Geometric concepts (18Q) 

♦ Instructional technology (3Q) 


III. Instructional 

Activities in Mathematics 


• Homework (10Q) 

• % of mathematics instruction time (12Q) 

• % of problem-solving activities (8Q) 

• % of pairs or small groups work (6Q) 

• % of use of hands-on materials (6Q) 

• Use of calculators, computers, etc. (1 IQ) 

• Assessments (8Q) 

• Instructional influence (10Q) 

• Classroom instructional prep. (18Q) 

• Teacher opinions (18Q) 

• Professional Development (14Q) 

• Formal course preparation (3Q) 

• Teacher Characteristics (8Q) 


Total 


• 155 questions 
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For example, Section I requests the teacher’s most recent unit of mathematics 
instruction and instructional time distribution, as well as school and classroom 
information. 

Figure 1. Sample Questionnaire in Section I 

What percent of mathematics instructional time was spent on the following activities? 



Enter the percentage of time for each item in the box provided, so that items 9-16 total 100%. Then use 
the scale to code your response (rounded to the nearest 10%) for each item on the answer sheet. 



- 


% 


None 


10 


20 


30 


40 


50 


60 


70 


80 


90+ 


9 Management or administrative routines. 
























interruptions, and other non-instructional 
activities 






© 


© 


© 


© 


© 


© 


© 


© 


® 


10 Whole class lecture or class discussion 






© 


© 


© 


© 


© 


© 


© 


© 




1 1 Individual student work (e.g. completing 
exercises, reading textbook) 




® 


© 


© 


© 


© 


© 


© 


© 


© 


® 


12 Small group work 




® 


© 


© 


© 


© 


© 


© 


© 


© 


® 


13 Field study or out-of-class investigation. 




® 


© 


© 


© 


© 


© 


© 


© 


© 


® 


14 Student demonstrations or presentations. 




® 


© 


© 


© 


© 


© 


© 


© 


© 


® 


1 5 Review or work on homework during class. 




® 


© 


© 


© 


© 


© 


© 


© 


© 


® 


16 Test or quiz 




® 


© 


© 


© 


© 


© 


© 


© 


© 


® 



100% (Note: Total should sum to 100) 



Section II requests information regarding topic coverage and teacher’s expectations 
for students in the target mathematics class for the current year. It is not intended to 
reflect any recommended or prescribed content for the grade level. For middle school 
mathematics, six topic areas are detailed by multiple sub-topic areas as shown in 
Figure 2. 

To complete this section, the teacher identifies topic/sub-topic areas covered in 
his/her mathematics class, using “time on topic” column (0= none- not covered, 1= 
slight coverage- less than one class/lesson, 2= moderate coverage- one to five 
classes/lessons, 3= sustained coverage- more than five classes/lessons). Then the 
teacher indicates the relative emphases of each student expectation for every sub- 
topic taught using scale bubbles for six categories: memorize, understand concepts, 
perform procedures, analyze/reason, solve novel problem, and integrate). Four scale 
bubbles indicates: 0= no emphasis, l=slight emphasis- accounts for less than 25% of 
the time, 2=moderate emphasis- accounts for 25% to 33% of the time, 3= sustained 
emphasis- accounts for more than 33% of the time. 
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Figure 2. Sample Questions for Mathematics Topic Coverage and 
Teacher’s Expectations for Students 



SECTION II 

Time on Topic Middle School Math Topics Expectations for Students in Mathematics 
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Concepts Procedures Reason Problems 
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Whole numbers 
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Ratio, proportion 
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Patterns 
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Real numbers 
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Exponents, scientific notation 
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Factors, multiples, divisibility 
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Odds, evens, primes, composites 
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Estimation 
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Order of operations 
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Relationships between operations 
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Mathematical properties (e.g. distributive 
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Section III asks 13 types of questions regarding instrucational practice. Figure 3 
illustrates a sample question regarding “Instructional Activities in Mathematics.” 

Figure 3. Sample Questions about Instructional Activities 

INSTRUCTIONAL ACTIVITIES IN MATHEMATICS _ 

Listed below are some questions about what students in target class do in mathematics. For each activity, 
pick one of the choices (0,1, 2, 3) to indicate the percentage of instructional time that students are engaged in 
the activity identified. Please think of an average student in this class, in responding. 



What percentage of mathematics instructional time in this target class do students: 



■ NOTE: No more than two * 3*8 , or four '2‘s should be circled for any given set of items. 1 




None 


Less than 25% 


25% to 33% 


More than 33% 


27 Observe the teacher demonstrate how to do a 
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procedure or solve a problem. 
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28 Read about math-related topics in books, 
magazines, articles 

29 Collect and/or analyze data. 
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30 Maintain a portfolio of their own work. 

31 Use hands-on materials/manipulatives (e.g. 
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counting blocks, geometric shapes, algebraic tiles, 
etc.). 
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32 Engage in mathematical problem solving (e.g. 










computation, story-problems, and mathematical 
investigations). 
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33 Students take notes. 
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34 Work in small groups 
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35 Work on an assigned mathematics project at 
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home or away from school. 
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36 Use computers, calculators, and/or other 
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technology to learning mathematics. 
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37 Work individually on assignments. 
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38 Take a quiz or test 
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CHAPTER III 



SURVEY RESULT 
SUMMARY AND ANALYSIS 



111.1 Overview 

State and national standards in math and science provided the constructs for-the 
survey design, item writing, and for reporting results. The standards used in design 
and reporting were selected by the CCSSO/NISE study team with a panel of 
mathematics and science education leaders from states. Not all topics and standards 
that provided the structure for the survey questions are reported here. This report is 
an initial analysis to provide a picture of some of the practices and curriculum 
content. Further reporting and analyses will be provided in subsequent reports. 

The profiles that follow are based on completed teacher surveys from 71 teachers of 
mathematics and 73 teachers of science in four USI sites: Baltimore, Dallas, Detroit, 
and Phoenix. A total of 58 schools were included in the 1999 survey, and a total of 
232 teachers were given surveys from their school contact persons. The total 
response rate from the four sites was just over 50 percent. 

The data presented in this report are not from a random selection of schools and 
classrooms in districts, and thus are not district-representative. The schools are a 
purposive sample of schools in the four selected districts, based on the school’s 
degree of invojvement in the USI project. The collected data offer sufficient numbers 
of responses to give meaningful statistics and analysis of differences between 
Implementation and Comparison schools. (Note: Baltimore surveys were not 
identified according to school designation, and the data are reported only for the 
summary graphs.) 

In all four districts, the elementary science surveys were completed by teachers of 
grade 5, and middle level classes were teachers of grade 8. The elementary 
mathematics surveys were completed by teachers of grade 4 in all four districts; while 
the middle level mathematics surveys were from teachers of grade 7 in Phoenix and 
teachers of grade 8 in Baltimore, Detroit and Dallas. 

The survey results are compiled and analyzed in two parts: The first part presents 
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“Item Profiles,” which display data on instructional practices used in math and 
science curriculum as well as teacher preparation. The second part presents “Content 
Maps” and “Content Graphs.” Each chart has a text page with data interpretation. 

111.2 Selected Item Profiles for Mathematics and Science 

The theme or concept that is being reported and analyzed in each data chart from the 
1999 Survey of Enacted Curriculum is based on a standard for learning in 
mathematics or science or a common goal of the USIs. Table 4 outlines the data 
charts presented in the report for mathematics and science. These 20 charts report on 
results for classroom instructional practices, teacher preparation, and subject content 
taught in classrooms. The charts and data presentations are intended to illustrate 
some of the findings that can be analyzed with surveys of enacted curriculum. 



Table 4. Selected Item Profiles for Mathematics and Science 



Chart 

No. 


Mathematics 


Chart 

No. 


Science 


1 


Use of Class Time in Mathematics 


2 


Use of Class Time in Science 


3 


Problem Solving Activities in Mathematics 


4 


Active Learning in Science 


5 


Small Group Work in Mathematics 


6 


Small Group Work in Science 


7 


Assessment Strategies in Mathematics 


8 


Assessment Strategies in Science 


9 


Use of Educational Technology in 
Mathematics 


10 


Use of Educational Technology & Lab 
Equipment in Science 


11 


Professional Development in Mathematics 


12 


Professional Development in Science 


13 


Teacher Course Taking in Mathematics 


14 


Teacher Course Taking in Science 


15 


Middle and Elementary School 
Mathematics Content Maps 


18 


Middle and Elementary School Science 
Content Maps 


16 


Middle School Mathematics Content 
Graphs 


19 


Middle School Science Content Graphs 


17 


Elementary School Mathematics Content 
Graphs 


20 


Elementary School Science Content Graphs 



Item Profile Format 

Item Profiles are used to report survey results in Charts 1-14. Each chart reports the 
total results by grade level for the four districts, and elementary and middle grades 
results by Implementation vs. Comparison school surveys for three districts. The 
vertical line indicates the item mean and shaded 
area shows the range of responses from -1 to +1 
standard deviation from the mean, which generally 
includes two-thirds of the survey responses. 
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Charts 1 & 2: Use of Class Time 

Teachers were asked to allocate 100% of class time during the most recent 
instructional unit into 9 categories of how time was used, both instructional and non- 
instructional activities. The purpose of this section of the survey is to obtain a broad 
picture of differences across the sample of classes and schools in approaches to 
instruction. In Chart 1 we present the data from all 9 possible responses for 
mathematics classes, and Chart 2 shows data for science classes. Below we highlight 
some of the findings from data in the charts: 

Mathematics: 

• Three kinds of classroom practices were reported most often: small group work, 
individual student work, and whole class lecture/discussion each averaged 20% 
of class time for both elementary and middle grade classes. There is wide 
variation in teacher responses, with a significant group of classes below 10% of 
time on each type of instruction, and a significant group over 35% of time. 

• N on-instructional time averages 10% at both grade levels. Review and 
homework activities also take about 10% of time on average. 

• Classes in high Implementation schools in the USI were similar to those in 
Comparison schools. In middle schools we can note more whole class lecture or 
discussion in Implementation classes, along with less time in small group work, 
more use of student demonstrations/presentations, and more time on tests or 
quizzes. 

Science: 

• Work with hands-on or laboratory materials, was the largest activity, with an 
average of 25% of time in middle and elementary classes. This activity is clearly 
advocated by standards and systemic reforms. 

• Non-lab small group work, individual student work, and whole class lecture/ 
discussion each covered from 10 to 20% of time each. Class management and 
interruptions took 1 0 to 20% of class time. Slightly more individual work in 
class and lecture/whole class discussion were used in elementary classes than in 
middle grade classes. 

• In elementary schools, teachers in high Implementation schools reported an 
average of 5% more time spent on active science activities (indicated by student 
work with hands-on, lab materials). Teachers in Implementation middle schools 
reported less time on review or homework in class slightly less time on 
management/interruptions. 
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Chart 1 

Use of Class Time 

During Most Recent Unit of Instruction in Mathematics 




What percent of mathematics instructional time in this unit was spent on the following? 
(NOTE: Responses should sum to 100.) 



Administrative routines, interruptions, and 




other non-in struct! on a 1 activities. 




Whole class lecture or class discussion. — j 

I 




Individual student work (e.g. completing 




exercises, reading textbook. 




Small group wcrk. 
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Field study or out-of-class investigation. 
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Student demonstrations or presentations. 
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Review or work on homework during class. 
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Chart 2 



Use of Class Time 

During Most Recent Unit of Instruction in Science 



Legend 

Mean 


Four District USI Sample 


By Grade Level 


Elementary 


Middle Sch. 


-1 StD +1 StD 


si Middle (35) 

K:Vy Elementary (38) 


HI Implementation (10) 
SR Comparison (18) 


SSI lmplementation(12) 
bJ Comparison (11) 



How many class periods did your most 
recent science instructional unit cover? 
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Response Codes: 



0 = 1-2 1 =3-5 2=6-10 3=11-15 

4 = 16-20 5 = 21 or more 



What percent of science instructional time in this unit was spent on the following? 
(NOTE: Responses should sum to 100.) 



. ] 

Administrative routines, interruptions, and j 


i 




other non- instructional activities. 
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Whole class lecture or class discussion. - 
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Charts 3 & 4: Problem-Solving Activities during Mathematics 
Instruction; Active Learning in Science 

Mathematics: 

Reasoning and Problem Solving is a cross-cutting standard for mathematics learning 
set out by NCTM in the Curriculum and Evaluation Standards (1989). The survey 
included a set of items on practices related to teaching reasoning and problem solving 
that taken together provide a reliable index of the degree to which teachers are 
focusing math instruction to student learning towards this standard. The broad term 
“problem solving” can be misinterpreted or used by teachers to mean very different 
kinds of instruction. The seven individual items graphed in Chart 3 provide specific 
information on the kinds of activities students are doing in class. Several specific 
results can be highlighted. 

• The seven activities in problem solving/reasoning gained a similar level of 
response from teachers, indicating that all of these activities are being used in 
most classes. Five class activities have slightly higher responses (about 15% of 
time), at both elementary and middle level classes: complete computation 
exercises , solve word problems , apply math to real world problems , make 
estimates/predictions, and analyze data . 

• There are several differences in problem solving activities. Elementary students 
in Implementation schools spend less time completing computational problems 
or solving word problems from a text or worksheet , and both elementary and 
middle school students in Implementation schools spend more time analyzing 
data to make inferences or draw conclusions . 

Active Learning in Science: 

The survey included a set of questions asking more details about the kinds of active, 
hands-on learning present in science class. Active Learning is one of the central 
concepts in state and national standards for student learning in science which were 
used to construct the survey. The data in Chart 4 show results for a range of specific 
actions when students are doing an experiment or laboratory activity. 

• In an experiment or investigation, students would be expected to be doing a 
number of the activities in combination or sequence. Three activities are 
reported more often: use science equipment, follow step-by-step directions and 
make tables , graphs or charts. Teachers report that students are less often 
changing something in an experiment to see what happens or designing an 
experiment. 

• We can note in column two that elementary students in Implementation schools 
are less likely to follow step-by-step directions and more likely to change 
something in an experiment to see what will happen , which suggests more 
student directed activity in science class. 
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• We can also note in column three that middle grade classes in Implementation 
schools spent more time using science equipment and tools in experiments or 
investigations. They also spent more time collecting data and designing ways to 
solve a problem, than students in Comparison schools, but they were less likely 
to make predictions, guesses or hypotheses, or to draw conclusions from science 
data. 



Chart 3 

Problem Solving Activities During Mathematics Instruction 



Legend 

y Mean 

-1 StD +1 StD 



Four District USI Sample 


By Grade Level Elementary 


Middle Sch. 


IS!' Middle (32) | EH Implementation (11) 

121 Elementary (39) rv; Comparison (17) 


EH Implementation (11) 
[. Comparison (15) 



When students are engaged in problem-solving activities, what portion of that time are students engaged in the following? 



Complete computational exercises or 
procedures from a text or a worksheet. 



Solve word problems from a textbook or 

worksheet. 



Solve novel mathematical problems. 



Write an explanation to a problem using 
several sentences. 



Apply mathematical concepts to real or 
simulated "real-world” problems. 



Make estimates, predictions, guesses, or 

hypotheses. 

Analyze data to make inferences or draw 

conclusions. 
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16 



26 



Chart 4 



Active Learning in Science 



Legend 
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Mean 
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Four District USI Sample 


By Grade Level 


Elementary 


Middle Sch. 


m Middle (35) 

Elementary (38) 


E3 Implementation (10) 
£J. Comparison (18) 


[23 Implementation (12) 
\j2. Comparison (11) 



When students are engaged in laboratory activities, investigations, or experiments, what portion of that time are students 
engaged in the following? 



Follow step-by-step directions. 
Use science equipment or measuring tools. 

Collect data. 



Change something in an experiment to see 
what will happen. 



Design ways to solve a problem' 
Make predictions, guesses, or hypotheses. 

Make tables, graphs, or charts. 

Draw conclusions from science data. 






m 



0% 20% 40% 0% 20% 40% 
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Charts 5 & 6: Small Group Work 

Teachers reported in Use of Class Time that small group work was a major 
instructional activity in both mathematics and science. Instruction directed toward 
scientific inquiry and student skills in reasoning and problem solving has often been 
organized so that students work together in pairs or small groups. At the same time, 
student work in small groups might have other learning objectives or simply be an 
organizational scheme for the classroom. 

The Survey included a set of five items that provide more detailed information about 
student roles and activities in small group work in class. The results from these items 
in Charts 5 and 6 provide a better picture of how small groups operate in math and 
science instruction, and they show differences across the study sample. 

Mathematics 

• The three items with the highest response in class time were: students talk about 
ways to solve math problems (30% of time), complete written assignments , and 
review assignments or problems. The first activity is often described as a reform- 
strategy for students to work together in discovering approaches to solving 
problems, and we see that classes in High Implementation schools spend more 
time with students talking about ways to solve problems. The other two small 
group activities indicate fairly traditional reasons for having students work in 
groups. 

• Student small group work on longer-term projects and on writing projects 
comprise 10 to 15% of time in class, at both elementary and middle grades levels. 
These uses might be viewed as instructional reforms in class activities, but the 
rates of use are similar in Implementation and Comparison schools. 

Science 

• Write results or conclusions of a laboratory activity had the highest response in 
class time in both elementary and middle schools (about 22% percent of time). 
This may reflect the importance of students being able to explain their work, a 
common assessment measure. In elementary schools, this activity, along with 
talk about ways to solve novel problems had the most variations in responses 
(from 5% to over 40% of class time), especially in high Implementation schools. 

• High implementation elementary schools spent less time (about 12%) on review 
assignments and problems than comparison schools (close to 20%). In middle 
grades of Implementation schools students spent less in small groups working on 
written assignments or working on longer assignments or projects than students 
in Comparison schools. 

• At the middle school level the two items with the greatest variation in responses 
between high Implementation and comparison schools are complete written 
assignments and work on longterm projects. The High implementation schools 
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reported more time spent on work on long term project (almost 20% vs. less than 
15%) and less on complete written assignments (15% vs. 20%). 



Chart 5 

Small Group Work in Mathematics 



Legend 



Mean 



-1 StD 
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Four District USI Sample 


By Grade Level 


Elementary Middle Sch. 


E3 Middle (32) 

1 • Elementary (39) 
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m Implementation (11) j H Implementation (11) 
^Comparison (17) Comparison (15) 



When students in the target class work in pairs or small groups as part of mathematics instruction, what percentage 
of that time do students: 



Talk about ways to solve mathematics 
problems 



Complete written assignments from the 
textbook or worksheets. 



Work on an assignment, report, or project 
that takes longer than one week to complete. 



Work on a writing project where group 
members help to improve each others’ (or the 

group’s) work. 

Review assignments, problems, or prepare for 

a test or quiz. 




0% 20% 40% 0% 20% 40% 
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Chart 6 



Small Group Work in Science 
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Four District USI Sample 


By Grade Level 


Elementary 
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Elementary (38) 


□ Implementation (10) 
L; Comparison (18) 


□3 Implementation (12) | 
LI J Comparison (11) 



When students work in pairs or small groups as part of science instruction, what percentage of that time do 
students: 



Talk about ways to solve science problems. 



Complete written assignments from the 
textbook or worksheets. 

Write results or conclusions of a laboratory 

activity. 



Work on an assignment, report, or project 
that takes longer than one week to complete. 

Work on a writing project where group 
members help to improve each other’s (or the 

group’s) work. 

Review assignments, problems, or prepare for 

a test or quiz. 
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Charts 7 & 8. Assessment Strategies in Mathematics and Science 

Rather than relying on a single type of assessment, such as paper-and-pencil tests 

comprised of objective items or routine procedural problems, mathematics and 

science teachers are being encouraged to make use of a variety of assessment 

strategies. In part, this is to increase the validity of the inferences that teachers can 

then make about student learning. Using multiple sources of evidence allows the 

strengths in one type of assessment to compensate for weaknesses in another. But to j 

what extent are mathematics and science teachers moving beyond a reliance on a 

single type of assessment, and what other strategies are they using? The results in 

Charts 7 and 8 illustrate how Survey data can be used to find out. 

Mathematics Item Profiles: 

The responses from teachers of mathematics on the assessment questions are 
displayed in Chart 7. The highest frequency of responses on strategies were short 
answer , extended response, performance tasks , and observation : 

• Short answer: A common form of assessing student knowledge in class is short 
answer questions, such as asking students to perform a mathematical procedure. 

Students are given this type of assessment, on average, about once a week. This 
is true across grade levels and with both Implementation and Comparison 



• Extended response : Elementary Teachers in High Implementation schools are 
significantly more likely than Comparison school teachers to ask students to 
explain or justify their answers (1-3 times/week). 

• Performance tasks are used to a significantly greater degree by Implementation 
teachers than Comparison teachers, both at the elementary and the middle grade 
levels. 

We should note that objective questions are used only monthly on average, although 

a significant portion uses them weekly. 

Science Item Profiles: 

• Portfolios : There are large differences in the use of several assessment strategies 
in science, and especially wide variation in the use of portfolios. Some teachers 
are not using portfolios at all, while some elementary school science teachers are 
using portfolios in assessment 1 to 3 times per month. The average for the 
middle school science teachers in Implementation schools is 1 to 4 times per 
year. Implementation elementary school teachers have greater use of portfolios. 

• Performance tasks use in science assessment averages about the same among 
Implementation and Comparison classes, both at the elementary and the middle 



schools. 
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grade levels. There is more variation in use in Implementation schools. 

• Extended response : Teachers report that students are asked to write explanations 
for answers, on average, about 1 to 3 times per month, about as often as objective 
or short answer items. All three items are used less often in elementary schools. 



Chart 7 

Assessment Strategies in Mathematics 
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How often do you use each of the following strategies when assessing students in this mathematics class? 



Objective items (e.g., multiple choice, 
true/false). 

Short answer (e.g., fill-in-the-blank). 

Extended response items for which 
student must explain or justify answer. 

Performance task or events (e.g., 
hands-on activities.) 

Individual or group demonstration, 
presentation. 

Mathematics projects. 



Portfolios. 



Systematic observation of students. 





Response Codes: 



0 = None 1= 1-4 times/year 2-1-3 times/month 

3 = 1-3 times/week 4= 4-5 times/week 
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Chart 8 



Assessment Strategies in Science 
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How often do you use each of the following strategies when assessing students in this science class? 



Objective items (e.g., multiple choice, „ 
true/false). 

Short answer (e.g., fill-in-the-blank). 

Extended response items for which 
student must explain or justify answer. 

Performance task or events (e.g., 
hands-on activities.) 

Individual or group demonstration,.. 

presentation. 

Science projects. 
Portfolios. 

Systematic observation of students." 
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Charts 9 & 10: Use of Educational Technology and Lab Equipment. 

An important indicator of active, inquiry-based methods of teaching science and 
math, as well as school system capacity for supporting this approach, is the 
availability and use of educational technology and laboratory equipment. Science 
and mathematics standards advocate learning to apply knowledge to real problems 
and gain skills that will be used outside of school. Science and mathematics uses in 
careers now involve computers, calculators, and a variety of simple and complex lab 
equipment. Thus, a key component of the survey of enacted curriculum is questions 
concerning the use of equipment and technology in teaching science and math. 

Mathematics: 

Chart 9 illustrates how the data can be reported to examine several kinds of questions 
concerning availability and instructional uses. 

• Calculators : Teacher reports on students’ use of calculators show that middle 
grades classes average use is less than weekly, although many classes use them 
weekly. Use in elementary classes differs widely, from rarely used to weekly use, 
and classes in Comparison schools had slightly greater use. Graphing calculator 
use varies widely. Middle grades classes differ from rarely used to almost 
weekly use. 

• Frequent Uses: The survey data show that the most frequent uses of educational 
technology in math instruction are: (a) learning facts or practicing procedures 
(20% of time average), (b) displaying/analyzing data (18% middle grades), 
testing (18% middle grades), and (d) individualized instruction (20% 
elementary). 



• Calculators or computers: Education technology is used most often in science 
class for learning facts (30% in Implementation schools) and for retrieving 
information or displaying/ analyzing data (28%). We can also note that these uses 
vary widely in both groups of schools, from 0 to 40% of time, indicating that 
teachers make very different uses of technology in instruction. 

• Lab Equipment: We highlight two different types of equipment in classrooms. 
First, running water (a traditional indicator of lab capacity) is not available in 
about one-third of elementary classes, and it is only rarely used in the average 
elementary class. The average middle grade science class uses running water less 
than monthly, although classes in Implementation schools average monthly to 
weekly use. Second, "high-tech" approaches to experimentation use computer 
lab interfacing devices. Our survey with urban schools showed that about a third 
of classes did not have these devices and the average class had access, but rarely 
used them. 



Science: 
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Use of Calculators, Computers & Educational Technology in 

Mathematics 
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Indicate how often the average student uses each of the following types of equipment in this mathematics class: 



Calculators 
Graphing calculators 
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When students are engaged in activities that involve the use of calculators, computers, or other educational technology as 
part of mathematics instruction, what percentage of that time do students: 



Learn facts or practice procedures."”” 

Use sensors or probes.- 

Collect data or information (e g., using the_ 

Internet). 

Display and analyze data.- 

Develop geometric concepts. 

Take a test or quiz.- 

Use individualized instruction ortutoriaL. 

software. 




a 



t 1 1 r 



0% 20% 40% 





0% 20% 40% 




35 



25 



Chart 10 



Use of Educational Technology and Lab Equipment in Science 
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Indicate how often the average student uses each of the following types of equipment in this science class: 



Computer/lab interfacing devices. 






Running water in laboratories. 



Electrical outlets in laboratories. 



Other lab equipment (e.g., scales, balances). 



Response Codes: 
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Charts 11 & 12. Professional Development 



Teachers were asked to report the total number of hours spent on professional 
development or in-service in mathematics education during the past year, and to 
report their response to the kinds of activities in which they participated. The Survey 
results for the four districts are in Charts 1 1 and 12. 

Mathematics: 

• Middle grade teachers received an average of almost 20 hours of professional 
development in both teaching mathematics and study of math content. 

Elementary teachers averaged 10 hours in each area. Teachers in Implementation 
schools received significantly more professional development at both elementary 
and middle grade levels. 

• The highest positive responses from teachers concerning their professional 
development activities were for: implementing new curriculum (elementary and 
middle), new methods of teaching (e, m), meeting needs of all students (e, m), 
implementing standards (e), in-depth study (m), and education technology (m). 

Science: 

• Middle grades science teachers received an average of almost 20 hours of 
professional development in both methods of teaching science and study of 
science content. Elementary teachers received about 10 hours less professional 
development in each area. Teachers in Implementation middle grade schools 
received more science education professional development than those in 
Comparison schools. 

• The professional development activities of most teachers in the past year 
included student assessment , in-depth study , new teaching methods , 
implementing new curriculum , and implementing standards. Responses were 
most positive to new teaching methods (middle grades), implementing new 
curriculum (e, m), in-depth study (e, m), implementing standards (m), and 
meeting needs of all students (e, m). 
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What is the total amount of time in the last twelve months that you spent on professional development or in- 
service activities in the following categories? 



In-depth study of mathematics content. 



Methods of teaching mathematics. 



Hours: 0 <6 6-15 16-35 >35 0 <6 6-15 16-35 >35 0 <6 6-15 16-35 >35 




For each of the following professional development activities that you participated in during the last 12 months , 
what best describes the impact of the activity ? 







Did not 
participate 


LiWe or 
no impact 


Trying 
to use 


Changed 
my practice 
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Chart 12 



Professional Development in Science 



Legend 

Mean 



-1 StD +1 StD 



What is the total amount of time in the last twelve months that you spent on professional development or in- 


service activities in the following categories? 










* « > * | ■ \ [ ] 




In-depth study of science content. 


; I ...i I 


l 8 l ; : 


1 T 1: 




* 


1 


Methods of teaching science. 


: r ' « ~ 1 : 


r « 1 : 


r 1 |: 


1 


i 


8 


Hours: C 


) <6 6-15 16-35 >35 C 


1 <6 6-15 16-35 >35 i 


0 <6 6-15 16-35 >35 



Four District USI Sample 


a 


By Grade Level 


Elementary 


Middle Sch. 


□ Middle (35) 

Elementary (38) 


1 i Implementation (10) 
Comparison (18) 


i □ Implementation^) 
Comparison (11) 
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Charts 13 & 14. Teacher Course-Taking Preparation 

Charts 13 and 14 provide important data on the extent of preparation of teachers in 

the subjects of mathematics and science they are teaching. The Survey asked 

teachers to report the total number of courses they had completed in undergraduate or 

graduate education. In the four USI districts we find the following summary results. 

Mathematics: 

• The average middle grades math teacher in these districts has taken 4 advanced 
math courses (calculus or higher), and 3 or 4 refresher courses (algebra, 
geometry, i.e., high school math courses). A significant portion of have no 
advanced or refresher courses. The average middle grade teacher has taken 5 or 6 
math education courses. 

• The average elementary teacher in these districts has taken 2 advanced math 
courses (calculus or higher), and 3 to 4 refresher courses. A significant portion 
of the teachers have no advanced or refresher courses. The average elementary 
teacher has taken 7 to 9 math education courses. 

• Elementary teachers in the Comparison schools have more preparation in 
mathematics and math education, but middle grades teachers in the 
Implementation schools have slightly more preparation in mathematics and math 
education. 

Science: 

• The average middle grades science teacher in these districts has taken 3 courses 
in life science or biology, 1 to 2 courses in physical science, and 3 in earth 
science, and 9 or 1 0 courses in science education. 

• The average elementary teacher in these districts has taken 1 or 2 life science 
courses, less than 1 physical science course, and 1 to 2 courses in earth science, 
and 3 to 4 courses in science education. 

• Elementary teachers in Implementation schools are more likely to have taken at 
least one physical science course, and have taken more science education courses 
than teachers in Comparison schools. 
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Chart 13 



Teacher Course-Taking in Mathematics and Mathematics Education 
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Chart 14 

Teacher Course-Taking in Science and Science Education 
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Indicate the number of quarter or semester courses that you have taken at the undergraduate or graduate level in 
each of the following areas: 



Biology/Life Sciences. 
Physics/Chemistry/Physical Science. 
Geology/Astronomy/ Earth Science. 

Science Education. 
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III. 3 Content Maps and Content Graphs for Mathematics and Science 



Data on the mathematics and science content taught in class have been compiled and 
visualized using “content maps,” and “content graphs.” A content map, as shown in 
Figure 4, provides a two-dimensional representation of instructional content using a 
surface area chart, which results in a graphic very similar to topographical maps. The 
grid overlaying each map identifies a list of topics areas (indicated by horizontal grid 
lines- O below) and six categories of cognitive expectations for students (indicated 
by vertical grid lines- © below). The intersection of each topic area and category of 
cognitive expectation represents a measurement node. A Content Map is an excellent 
instrument for general overview using a topographic perspective. 



Figure 4. A Sample Mathematics Content Map 




Number Sense 



Operations 



Measurement 



Pre-Algebra 



Geometric 

Concepts 



© 





32 



42 




For more detailed node-by-node comparison, a Content Graph is presented in a 
matrix format combining multiple bar graphs on each node with scales. Each bar 
graph compares Implementation and Comparison schools. Each Row Total shows a 
mean value and a range of one standard deviation for each topic area. Likewise, each 
Column Total shows a mean and standard deviation for six categories of cognitive 
expectations for students. 

Content maps and content graphics are both generated from the teacher responses 
across all four USI school districts. The teacher responses are aggregated and 
reported by grade level (elementary, middle) and Implementation vs. Comparison 
schools. 

The following section provides six Content Maps and Content Graphs for 
Mathematics and Science as follows: 

Chart 15. Middle and Elementary School Mathematics Content Maps 

Chart 16. Middle School Mathematics Content Graphs 

Chart 17. Elementary School Mathematics Content Graphs 

Chart 18. Middle and Elementary School Science Content Maps 

Chart 19. Middle School Science Content Graphs 

Chart 20. Elementary School Science Content Graphs 

Appendix D presents 20 charts for Item Profiles and Content Maps for further 
analysis. 

• Class Description 

• Use of Homework 

• Instructional Activities 

• Use of Hands-on Material in Mathematics 

• Student Reflection on Scientific Ideas 

• Influences on Instructional Practice 

• Teacher Readiness 

• Teacher Opinions 

• Grade 4 Mathematics Content Maps 

• Grade 8 Mathematics Content Maps 

• Grade 4 Science Content Maps 

• Grade 8 Science Content Maps 
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Mathematics Content 



Chart 15 provides a Cross-District Map for middle and elementary school 
mathematics by Comparison and Implementation schools. The map allows the reader 
to see the basic patterns of content topics by teacher expectations for student learning. 

Chart 16 and 17 provide Content Graphs for mathematics which allow more 
detailed analysis of each intersecting cell and the percentages for each row and 
column category. 

• Middle level math teachers reported over 20% of time each on number sense, 
algebraic concepts and geometric concepts. Measurement and data analysis 
were taught an average of 12-15% of time. Expectations focused on understand 
concepts and perform procedures (over 20%) and about 15% on 
analyze/reasoning and memorize. 

• Middle teachers in Implementation schools reported significantly more time on 
geometric concepts and data analysis, and less time on number sense. The 
implementation school teachers focused less on expectations for memorization, 
and place more emphasis on integrating concepts than the Comparison school 
teachers. 

• In elementary math, the topics in number sense, operations, and geometric 
concepts were taught an average of 20% of instructional time. The greatest 
intersection with expectations was with understand concepts and perform 
procedures. Solving novel problems and integrating math concepts were 
expected primarily with the topics in number sense and operations and geometric 
concepts. 

• The Implementation elementary school teachers reported more time on spent on 
operations than Comparison teachers. The other content categories have similar 
average time and variation between groups of schools. 
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Chart 15 



Middle and Elementary School Mathematics Content Maps 

Four USI District Sample: Comparison vs. Implementation 
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Chart 16 



Middle School Mathematics Content Graphs 

Four USI District Sample: Comparison (15), Implementation (11) 



Number 

Sense, 

Properties, 

Relationships 



Measurement 



Pre-Algebra 



Geometric 

Concepts 



Data Analysis, 

Probability, 

Statistics 



% 

10 

8 

6 

4 

2 

0 

10 

8 

6 

4 

2 

0 

10 

8 

6 



Understand Perform Analyze/ Solve Novel 
Memorize Concepts Procedures Reason Problems Integrate 



71" 



IE 






iGldna 









. . 1 
> 




. 










...j 

i 


! 




EDd 




r :U 




1 ! 
! 














4 

m 








E 

i 


| 








+W» ■■ 1 — < 


I L 





□::: 


CD 




















v- :: r 

. . 7 . . 1 T7 


T ' ' ■ - ! 


in ! - 


n ' 







j I 



£1 =□ 



% 



Row Totals 



50 
^40 
,30 
! 20 
10 
! ° 
j 50 
40 
| 30 
20 
10 L 
| 0 
150 | 
40 I 
■ 30 I 
20 

! io i 
o L 

50 I 
40 ; 
30 ■ 
20 L 
10 1 

! 0 1 

;50 
1 40 
130 
20 

10 ; 

0 



•5 



ii 



Column 

Totals 


50 

40 

30 

20 

10 




















| c- — 






Cl_. . . . 


F t 


.... ... p-, ., 


■ ■ ■ 43-7 


: : S\ 


... L f . 


M3 gp 1 


0 




i 


i EH" 


Memorize 


• -1 * 1 

Understand Perform Analyze/ 
Concepts Procedures Reason 


Solve Novel 
Problems 


1 Integrate 



[H Comparison 
□ Implementation 



Summary Legend 







+1 StD 




- 


Mean 






-1 StD 



O 

ERIC 



36 



Chart 17 



Elementary School Mathematics Content Graphs 

Four USI District Sample: Comparison (17) vs. Implementation (1 1) 
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Science Content 



Chart 18 illustrates a Cross-District Map for middle and elementary school science by 
Comparison and Implementation schools. The map allows the reader to see the basic 
patterns of content topics by teacher expectations for student learning. Charts 19 and 
20 are Content Graphs for science which show detailed analysis of the intersecting 
cells in Chart 18, and the percentages for each row and column category. 

• The middle school science content taught varied most between Comparison and 
Implementation schools with physical science. Average time spent by 
Comparison school teachers was about 38% and by Implementation school 
teachers about 22%. Teachers in Implementation schools, on average, spent more 
time on life science and chemistry, with wide variation in time spent on these 
subjects as well as in physical science. 

• Classes in Comparison middle schools had more emphasis on expectations of 
memorize and analyze information than those in Implementation schools, and 
wider variation in expectations for understand concepts and conduct 

experiments. 

• In elementary schools the greatest intersection of content and expectations was 
life science and understand concepts. There was not a correspondingly clear 
intersection in middle schools. 

• There was a marked difference between Comparison schools and Implementation 
schools in elementary science content taught. Implementation school teachers 
taught nature of science an average of 25% of time and life science an average of 
32% of time vs. Comparison school teachers’ average times of 1 0% and just over 
20% of time respectively. On the other hand, Implementation school teachers 
spent an average of over 20% of time on physical science vs. 10% for 
Comparison school teachers. 
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Chart 18 



Middle and Elementary School Science Content Maps 

Four USI District Sample: Comparison vs. Implementation 
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Chart 19 



Middle School Science Content Graphs 

Four USI District Sample: Comparison (13) vs. Implementation (13) 
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Chart 20 



Elementary School Science Content Graphs 

Four USI District Sample: Comparison (16) vs. Implementation (10) 
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CHAPTER IV 



CONCLUSIONS 



IV.1 Summary of Findings 

The initial report on enacted curriculum in USI sites provides two very important 
kinds of results. First, we demonstrate that the survey approach tried with four USI 
sites in 1999 can be used to analyze curriculum and teaching in classrooms, and that 
the analysis can be used across different classes, schools, and districts. The data 
reported from 1999 demonstrates that surveys with teachers will produce quantitative 
data that can be aggregated and reported in multiple sites. Second, the report 
demonstrates that a purposive sample of schools and teachers can be used to compare 
curriculum and instruction in schools with high implementation of systemic reform 
through USI with schools that have less implementation. 

Improvements can be made in the surveys, data collection, and analysis. Methods of 
obtaining lists of schools and teachers in advance will improve capacity for verifying 
sample selection, and improve follow-up efforts with teachers. Additional 
information was collected about schools in the sample that was not presented in this 
analysis. Based on initial response to this report, modifications can be made in the 
analysis plan, data profiles, and content maps, and other strategies for analysis and 
reporting can be developed. Subsequent interactions of the reports in this project 
will incorporate feedback and review suggestions. 

IV.2 Uses and Misuses of Data 

Data on enacted curriculum cannot itself provide a vision of quality education, but it 
can inform that vision by providing indicator measures to serve as guideposts and 
measuring sticks to determine where things are and where they need to go in order to 
move closer to the goals selected. 

By providing a broad selection of measures pertaining to content, pedagogy, climate, 
and background, survey data allows for concerned individuals and groups to 
construct their own individualized set of indicators for determining how local practice 
compares to their and others’ visions of quality practice. 
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The survey data also allows schools to determine estimates of time spent engaged in 
these various types of activities, as well as more descriptive information about how 
these activities are provided, which together form both an operational definition of 
desired practice as well as a description of current practice. 

While data from these surveys provide a wealth of valuable information about what 
goes on in classrooms, care must be taken not to over-interpret or even misinterpret 
the information provided from the surveys. 

Curriculum Content Data 

The content matrix portion of the Survey of Enacted Curriculum represents the most 
thorough and detailed approach to measuring classroom subject content that has been 
developed and tested using a teacher self-report methodology. The method is the first 
to effectively incorporate both content topics and expectations for learning in teacher 
reporting on what is taught in classes. This matrix approach is critical to analyzing 
mathematics and science curriculum and teaching in order to relate classroom content 
to state and national standards developed in the 1990s. Leading educators have long 
known that curriculum is not simply a listing of topics but rather includes what 
students are expected to learn about the topics and what skills and knowledge they 
gain about them. 

There are issues in obtaining valid, reliable data using the content matrix method. 
Teachers have to report their curriculum in terms of a common set of categories and 
time scales. It requires concentration and time to complete. As the surveys have 
been field tested, the design has been changed and simplified. The analysis of data 
from the USI surveys in 1999, and other teacher surveys recently conducted, indicate 
that the content matrix methodology does work. We are finding that a 
comprehensive picture of classroom curriculum across schools and districts can be 
obtained using this approach. 

Specific Classes 

Since the survey data represents teacher perceptions of classroom practice targeted to 
a particular class and group of students, it should be kept in mind when interpreting 
such data that teacher reports for a particular class may not represent that teacher’s 
practice across all classes and students. 

Socially Desired Responses 

In considering the limitations of the survey data, it should be noted that the accuracy 
of the data depends upon the accuracy of teachers’ perceptions and estimates. 
Accuracy can also be affected by perceptions of socially desirable responses, 
particularly if sanctions or rewards are commensurate with certain practices. 

Misuse of Data 

The primary misuse of data on the enacted curriculum or teaching practices is the use 
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of such data for accountability purposes. To be a useful tool for school improvement, 
this type of indicator data should stand aside from the accountability mechanisms 
developed by schools, districts, or states. In large part, this is due to the possibility 
that the diagnostic potential of such data would be compromised if used for 
accountability purposes because of the increased likelihood of teachers providing 
socially desirable responses to the survey questions. Further, the indicators that such 
measures provide should not be rigidly imposed as a definition of quality practice. 
Survey data on curriculum must be combined with local contextual information in 
order to provide a sufficient basis for making decisions about changing, rewarding, or 
sanctioning individual practice. For these reasons this indicator data is best conceived 
of as a diagnostic and not an accountability tool. 
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APPENDIX A: 



USI EVALUATIVE STUDY ABSTRACT AND 
YEAR 1 PROGRESS SUMMARY 



ABSTRACT 

The purpose of the study is to determine the impact of the National Science 
Foundation’s Urban Systemic Initiative (USI) program on student achievement and 
the learning infrastructure in urban school districts, and to develop an inferential 
causal model that relates the SI drivers and other key elements to the outcomes 
observed. Our study team will analyze the effectiveness of the USI program from 
five different perspectives: across all 21 USI sites; by USI cohort; longitudinally 
within sites; by comparison with 7 non-USI cities; and within a national context that 
includes other major reform efforts. 

The study team will develop a Key Indicator Data System (KIDS) to collect/ compile 
annual core data from the baseline year using both quantitative (K-l) and qualitative 
templates (K-2). Our study team will conduct an annual KIDS workshop designed 
for USI core data managers and evaluators to enhance data integrity and share 
expertise. Based on collaborative research agreements, we will receive SAT, AP, and 
ACT test results (as part of K-l data) directly from the Educational Testing Service, 
the College Board, and ACT, Inc., respectively, for the next three years. 

Each year, the team will conduct four site visits including interviews with USI 
leaders, focus groups, and school visits, to further explore detailed implementation 
issues. In addition, a modified version of the Council of Chief State School Officers’ 
Survey of the Enacted Curriculum will be administered to 80 selected math and 
science teachers within these four sites. Finally, we will gather and review 
information from national databases such as the Third International Mathematics and 
Science Survey (TIMSS), National Assessment of Educational Progress (NAEP), 
Goals 2000, Equity 2000, as well as the USI sites’ annual reports and relevant 
documents. 

The team has identified key research questions in eight areas mapped to the NSF 
drivers: (1) student outcomes, (2) curriculum and instruction, (3) assessment 
instruments, (4) professional development, (5) policy, (6) leadership, governance, and 
management, (7) convergence of resources, and (8) broad-based support for reform. 
We will use a variety of statistical analysis methods to validate the hypothesis that a 
well-implemented USI program has a positive impact on student outcomes. Further 
systemic analysis will explore the determinants of successful implementation of 
urban reform in mathematics, science, and technology education. 
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During the three-year study period of October 1998 to September 2001, we will 
produce a USI Annual Fact Book and USI Evaluative Study Report, both in print and 
in CD-ROM format, and a master data base to be available on the world-wide web. 
Beginning in Year 2 we will publish a series of newsletters and monographs detailing 
our research findings. 



YEAR 1 PROGRESS SUMMARY 

As the study has wrapped up its Year 1 effort, a few brief notes on our study progress 
are: 



• Study Monograph No. 1 What Matters in Urban School Reform : The first 
monograph has been published as an initial literature review relevant to our 
evaluative study. The review of selected literature examines the National Science 
Foundation’s standards-based systemic reform theory in the context of research 
reported by science, mathematics, and urban education policy experts. The 
review is specifically concerned with viewing evidence of relationships among 
identified change variables that are associated with fostering high achievement in 
mathematics and science of urban and underrepresented minority students. The 
authors indicate that four categories of variables are useful in studying student 
achievement: demographic information, test data, teacher development, and 
mathematics and science curriculum. The review revealed little researcher 
agreement regarding variable relationships that might predict a chain of influence 
from policy to classroom practice and finally to student performance. 

• Site visits: The four site visits conducted in Year 1 of the study served to validate 
the study’s approach to analyzing change in policy and practices across all sites. 
The evaluative study relies on written reports, data, and documents submitted by 
the sites to NSF or requested through the study. The site visits provided valuable 
depth of understanding supplementing the material described in written reports. 
We are building on the qualitative understanding gained from interviews, 
observations, and on-site analysis to develop a fair, reliable methodology for 
analyzing and coding the written material available from all the USI sites. In no 
case has any site visit led to findings that are opposite to or significantly different 
from what we learned through reading annual reports. 

• KIDS: The study team developed and implemented a Key Indicator Data System 
(KIDS) to collect and compile annual progress data including both quantitative 
(K-l) and qualitative data (K-2) from all 21 USI sites. KIDS focuses on student 
achievement data including mathematics and science enrollment and completion, 
mathematics and science assessment test results, SAT-I, AP, and ACT test 
results, as well as disaggregated demographics. KIDS data will provide a basis 
for causal inferential models focusing on the National Science Foundation’s six 
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systemic initiative drivers. KIDS data will also be cross-examined with this 
enacted curriculum survey results. 

• USI Fact Book I & II: Two USI Fact Books are in progress compiling both K-l 
and K-2 data for further analysis and visualization of impact data. Fact Book-I 
presents each USI site’s progress based on both quantitative and qualitative key 
indicators, and Fact Book-II presents overall program impact. 

• Cross-Site Qualitative Summary and Rubrics: Based on KIDS, our study team 
compiled the data into USI Cross-Site Qualitative Summary Tables and produced 
relevant Rubric scales which will be utilized to develop inferential models. 

• AP, SAT, and ACT: We have received five year AP, SAT, and ACT test result 
data from the College Board, Educational Testing Service, and ACT, Inc., 
respectively. We recompiled the 21 site reports’ data and transferred it to the K-l 
data templates. The compiled data is also being reformatted for SPSS analysis. 
Initial review of sites' AP and SAT data show upward trends in test takers; this 
will require more detailed analysis. 

• Enacted Curriculum Survey: The purpose of the Enacted Curriculum Survey is to 
analyze current mathematics and science instructional practices at the classroom 
level based on teacher responses to the questionnaires regarding: classroom 
practices, course-work background, professional development, curriculum 
influences, resources and equipment, teacher attitudes, and school conditions. 
Survey responses were received by mail. The survey results will be reported in 
relation to the analysis of findings from the four site visits conducted in 1999. 

The results will provide an analysis of the extent to which teaching practices and 
curriculum in schools with high implementation of the USI reforms differ from 
practices and curriculum in low implementation/ average schools. We will also 
analyze the extent to which the high reform schools exhibit teaching toward 
challenging content standards in mathematics and science. 

• SI Urban Study Web Site: Systemic Research, Inc. hosts a web site for the SI 
Urban Study Forum (www.siurbanstudy.org) to open a communication channel 
among researchers and evaluators currently working on the National Science 
Foundation’s Systemic Initiatives Research/ Evaluation/ Impact Studies. The 
Forum presents each study’s abstract, research design, methods, 
accomplishments, directory, and disseminated news and publications. 
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APPENDIX B: 

BRIEF DESCRIPTION OF FOUR USI SCHOOL DISTRICTS 



Baltimore 

Baltimore City Public Schools teamed up with Morgan State University to become 
one of the first cities to participate in the National Science foundation’s Urban 
Systemic Initiative. Baltimore Urban Systemic Initiative (BUSI) is creating a culture 
of math, science and technology success for all of its 104,000 students in 179 schools 
by changing the way these subjects are taught. The emphasis is on quality curricula, 
professional development, policy changes and instructional support. 

Highlights of BUSI include the aligning of curricula to national standards, the 
utilization of Instructional Support Teachers, and increased graduation requirements 
of three years each of mathematics and laboratory science. Thanks to BUSI, all 
schools have been provided with teaching tools from hands-on science materials at 
the elementary level , to microscopes at all middle schools, to renovations of the 
laboratories at the nine high schools. The results have been higher Maryland School 
Performance Assessment Program (MSPAP) and Grade 8 Maryland Functional 
Math Test (MFMT) scores, year round elementary school science instruction (from 
the pre USI practice of only one third year) and higher enrollment in mathematics and 
science classes. 

The system-wide changes BUSI initiated will continue to prepare Baltimore’s 
children for success long after the program period is over. 



Dallas 

The Dallas Systemic Initiative (DUSI) goal is that all Dallas Independent School 
District students will upon graduation be prepared to succeed in the scientific and 
technological workplace. The goal will be accomplished in three ways: (1) improve 
the scientific and mathematical literacy of all students, (2) provide mathematics and 
science fundamentals and (3) prepare an increasing number of students for careers in 
science, mathematics, engineering and technology. 

DUSI emphasizes six elements for success: (1) program leadership, (2) total quality 
management (3) integrated curriculum, (4) interactive learning, (5) teacher 
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enhancement, and (6) aligned assessment and accountability. The program targets all 
150,000 students in 215 schools. 

DUSI has allowed Dallas to implement the K-12 curriculum based on standards- 
based, nationally recognized programs. The curriculum is supported by ongoing 
professional development and educational technology in the classroom. 

DUSI has also been instrumental in forming new policies and directing resources 
towards mathematics and science education. Important partnerships have been 
forged with leading companies such as Texas Instruments, IBM and Mobil Oil 
Corporation and Higher Education Institutions including the University of Texas and 
Texas A & M University. 



Detroit 

The goals of the Detroit Urban Systemic Initiative is to (1) improve the mathematical 
and scientific literacy of all 177,000 students in 260 schools, (2) provide the 
mathematics and science fundamentals to permit all students to participate fully in a 
technological society, (3) enable a significantly greater number of students to pursue 
careers in mathematics, science and technology and (4) facilitate a district-wide 
climate of change that assures adoption and maintenance of strategies and programs 
assessable to all students. 

DUSI embraces the Constructivist approach to teaching and learning. Constructivist 
learning is not passive- students learn by constructing meaning from hands-on 
activities and by building on their own past experiences and knowledge. Teachers 
must adapt their instructional strategies to individual student learning styles. DUSI 
supports extensive professional development to assist teachers with the knowledge, 
techniques and confidence to implement the Constructivist methods. 

DUSI has also been instrumental in the development of curriculum standards, student 
support programs such as Saturday Academies and formation of business, higher 
education and community partnerships. 



Phoenix 

The vision of the Phoenix Urban Systemic Initiative (PUSI) is to create a culture for 
learning and change that enables all students to function successfully in the 
mathematical, scientific and technological oriented society of today. This vision is 
articulated by the PUSI’s goals: (1) to improve the scientific, mathematical and 
technological literacy of all students (2) to provide the mathematics and science 
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fundamentals which will permit all students to participate fully in a technological 
society, (3) to enable a significantly greater number of these students to pursue 
careers in mathematics, science, engineering and technology, and (4) to produce 
systemic reform in science, mathematics, and technology education through 
fundamental, comprehensive, and coordinated change which will result in improved 
outcomes for all students. 

PUSI is comprised of 92 public schools in the Phoenix inner city, Arizona State 
University, the Arizona Science Center, Maricopa County Community College 
District and state and municipal agencies and community based organizations 
managed by the Unitary Management Team. The primary focus is on curriculum 
reform and professional development of all mathematics and science teachers. The 
program is organized into six components (1) curriculum/instruction, (2) staff 
development, (3) instructional support, (4) cross-jurisdictional education linkages, (5) 
parental involvement and (6) business/industry and community linkages 
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APPENDIX C: 

INTERPRETING CONTENT MAPS 



Content maps provide a three-dimensional representation of instructional content using a surface area chart which 
results in a graphic very similar to topographical maps. The grid overlaying each map identifies a list of topics areas 
(indicated by horizontal grid lines; see O below) and six categories of cognitive expectations for students (indicated by 
vertical lines; see © below). The intersection of each topic area and category of cognitive expectation represents a 
measurement node (see © below). Each measurement node indicates a measure of instructional time for a given topic 
area and category of cognitive expectation based upon teacher reports. The resulting map is based upon the values at 
each of these measurement nodes. It should be noted that the spaces between each measurement node, that is the 
surface of the map, are abstractions and are not based upon real data, the image of the map is simply a computer 
generated graphic based upon the values for each intersecting measurement node. The map display is utilized to 
portray the third dimension (percent of instructional time; see © below) onto this grid utilizing shading and contour 
lines to indicate the percent of instructional time spent (on average across teachers) for each topic by cognitive 
expectation intersection. 



The increase (or decrease) in instructional time represented by each shaded band is referred to as the measurement 
interval (see © below). To determine the amount of instructional time for a given measurement node, count the 
number of contour lines between the nearest border and the node, and multiply by the measurement interval. 



The graphic at left below displays the three dimensional counterpart of the image represented by the content map 
displayed on the right. Both graphs indicate that Understanding Concepts related to Number Sense and Operations 
occupies the majority of time spent on grade four mathematics instruction (9% or more of instructional time over the 
course of a school year). 
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APPENDIX D: 

ADDITIONAL ITEM PROFILES AND CONTENT MAPS 



Chart D. 1 : Class Description - Mathematics 55 

Chart D.2: Class Description - Science 56 

Chart D.3: Use of Homework in Mathematics 57 

Chart D.4: Use of Homework in Science 58 

Chart D. 5: Instructional Activities - Mathematics 59 

Chart D.6: Instructional Activities - Science 60 

Chart D. 7: Use of Hands-on Materials in Mathematics 61 
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Chart D.9: Influences on Instructional Practice in Mathematics 63 

Chart D. 10: Influences on Instructional Practice in Science 64 

Chart D. 1 1 : Teacher Readiness (Part 1) - Mathematics 65 

Chart D. 12: Teacher Readiness (Part 2) - Mathematics 66 

Chart D. 13: Teacher Readiness (Part 1) - Science 67 

Chart D. 1 4: Teacher Readiness (Part 2) - Science 68 

Chart D . 1 5 : T eacher Opinions (Mathematics) 69 

Chart D. 16: Teacher Opinions (Science) 70 

Chart D. 1 7: Grade 4 Mathematics - Content Maps 71 

Chart D. 18: Grade 8 Mathematics - Content Maps 72 

Chart D.19: Grade 4 Science - Content Maps 73 

Chart D.20: Grade 8 Science - Content Maps 74 
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Chart D.1 

Class Description - Mathematics 
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Chart D.2 

Class Description - Science 
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Response Codes: 


0 = 10 or less 1=11 to 15 2= 16 to 20 

3 = 21 to 25 4 = 26 to 30 5 = 31 or more 


During a typical week, approximately 
how many hours will this class spend in 
scientific instruction? 

(Number of instructional hours) »> 
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What is the average length of each class 
period for this dass? 
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Response Codes: 


0 = N/A 1= 30 to 40 min. 2 = 41 to 50 min. 

3 = 51 to 60 min. 4=61 to 90 min. 5 = 91 to 120 min. 

6 = Varies due to block scheduling or integrated instruction. 
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Chart D.3 



Use of Homework in Mathematics 



Legend 
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By Grade Level 


Elementary j Middle Sch. 
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How many minutes does the typical 
student spend on a normal homework 
assignment? 
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Response Codes: 


0 = No homework assigned 1= < 15 minutes 2 = 15- 30 minutes 
3 = 31-60 minutes 4 = 61-90 minutes 5 = > 90 minutes 


How often do you usually assign 
mathematics homework? 
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Response Codes: 


0 = Never 1= < Once per week 2 = 1-2 times/week 

3 = 3-4 times/week 4 = Every day 
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Response Codes: 


0 = Never 1- Usually does not 2 = Usually does 3 = Always does 



What percentage of the time that students spend on mathematics homework do they: 



Do arithmetic computation or procedures. 



Show steps required in doing a procedure or 
solving an equation. 



Explain their reasoning or thinking in solving a 
problem. 

Work on a demonstration, presentation, or 
proof of their mathematics work. 



Collect data or information as part of 
mathematics homework. 



Write a report for a mathematics project. 
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Chart D.4 



Use of Homework in Science 



Legend 
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Four District USI Sample 


By Grade Level 


Elementary 
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□ Implementation (12) 
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How many minutes does the typical 
student spend on a normal homework 
assignment? 
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Response Codes: 


0 = No homework assigned 1=< 15 minutes 2 = 15- 30 minutes 
3 = 31-60 minutes 4 = 61-90 minutes 5 = > 90 minutes 


How often do you usually assign science 
homework? 
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Response Codes: 


0 = Never 1- < Once. per week 2 = 1-2 times/week 

3 = 3-4 times/week 4 = Every day 


Does homework count towards student 
grades? 
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Response Codes: 


0 - Never 1- Usually does not 2 = Usually does 3 = Always does 



What percentage of the time that students spend on science homework do they: 



Read about science in books, magazines, or 

articles. 

Answer questions from a science book or 
worksheet. 
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Solve science problems that involve | ipEZ 
computation. 



Revise and improve students’ own work. 
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Collect data or information about science. | 



Write about science. 
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Chart D.5 



Instructional Activities - Mathematics 



Legend 
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By Grade Level 
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What percent of mathematics instructional time in this unit was spent on the following activities? 
(NOTE: Responses should sum to 100.) 



Watch the teacher demonstrate how to do 
a procedure or solve a problem. 

Read about mathematics in books, 
magazines, or articles. 



Collect or analyze data. 



Maintain and reflect on a mathematics 
portfolio of their own work. 

Use hands-on materials or manipulatives 
(e.g. counting blocks, geometric shapes, 
algebraic tiles). 

Engage in mathematical problem solving (e.g. 
computation, story-problems, mathematical 
investigations. 



Take notes. 



Work in pairs or small groups. 



Do a mathematics activity with the class 
outside the classroom. 



Use computers, calculators, or other 
technology to learn mathematics. 



Work individually on assignments. 



Take a quiz or test. 
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Chart D.6 



Instructional Activities - Science 
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What percent of science instructional time in this unit was spent on the following activities? 
(NOTE: Responses should sum to 100.) 



Listen to the teacher explain something 

about science. 


1 U M 


Read about science in books, magazines, 
or articles. 


® | j 1 : 


Collect or analyze data. — 


■fl- : : : 


Maintain and reflect on a science portfolio of 

their own work. 


Bia : ; 

, ; 


Write about science. 
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Do a laboratory activity, investigation, or 
experiment in class. 


WM ; ; 


Watch the teacher give a 
demonstration of an experiment. 


; WM : i 
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Work in pairs or small groups (non- 
laboratory). 


WM : : 


Do a science activity with the class j 

outside the classroom or laboratory. 
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Use computers, calculators, or other 

technology to learn science. 
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Work individually on assignments. — 
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Take a quiz or test. 
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Chart D.7 



Use of Hands-on Materials in Mathematics 



Legend 
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When students are engaged in activities that involve the use of hands-on materials , what percentage of that time do 
students: 



Work with hands-on materials such as 
counting blocks, geometric shapes, or 
algebraic tiles to understand concepts. 



Measure objects using tools such as rulers, 
scales, or protractors. 



Build models or charts. 



Collect data by counting, observing, or 
conducting surveys. 



Present information to students concerning a 
mathematical idea or project. 
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Chart D.8 



Student Reflection on Scientific Ideas 
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When students collect information about science from books, magazines, computers, or other sources, what 
percentage of that time do students: 



Ask questions to improve understanding. 



Organize and display the information in tables 

or graphs. 

Make a prediction based upon the information 

or data. 



Discuss different conclusions from the 
information or data. 

List positive (pro) and negative (con) reactions 
to information. 

Reach conclusions or decisions based upon 
the information or data. 
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Chart D.9 



Influences on Instructional Practice in Mathematics 
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Indicate the degree to which each of the following influences what you teach in this mathematics class; 



Your state’s curriculum framework or content 
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Your district’s curriculum framework 
or guidelines. 
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National mathematics education standards. 
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Your experience in pre-service preparation. 
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Response Codes: 



0 = Not Applicable 1 = Strong Negative Influence 

2 = Somewhat Negative Influence 3 = Little or No Influence 

4 = Somewhat Positive Influence 5 = Strong Positive Influence 



) 



63 




73 



Chart D.10 



Influences on Instructional Practice in Science 
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Indicate the degree to which each of the following influences what you teach in this science class: 



Your state’s curriculum framework or content 

standards 



Your district’s curriculum framework 
or guidelines 



Textbook /Instructional Materials 



State test 



District test 



National science education standards „ 



Your experience in pre-service preparation 
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Response Codes: 



0 = Not Applicable 1 = Strong Negative Influence 

2 = Somewhat Negative Influence 3 = Little or No Influence 

4 = Somewhat Positive Influence 5 = Strong Positive Influence 
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Chart D.11 

Teacher Readiness (Part 1) - Mathematics 



Legend 
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mm 



-1 StD 



+ 1 StD 



Four District USI Sample 
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For the following items, please indicate how well prepared you are now to: 



Teach students with physical disabilities. 



Teach classes for students with diverse 

abilities. 



Teach mathematics to students from a 
variety of cultural backgrounds. 



Teach mathematics to students who have 
limited English proficiency. 

Teach students who have a learning 
disability which impacts mathematics 
learning. 

Encourage participation of females in 
mathematics. 



Encourage participation of minorities in 
mathematics. 
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Response Codes: 



0 = Not well prepared 1=Somewhat prepared 2= Well prepared 
3 = Very well prepared 
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Chart D.12 

Teacher Readiness (Part 2) - Mathematics 
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For the following items, please indicate how well prepared you are now to: 



Use/manage cooperative learning groups 
in mathematics. 
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Integrate mathematics with other subjects.— 

Implement instruction that meets 
mathematics standards. 
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Use a variety of assessment strategies 
(including objective and open-ended ~~ 
formats). 
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Teach estimation strategies. 








Teach problem solving strategies. 
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Select and/or adapt instructional materials 
to implement your written curriculum. 
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Teach mathematics with the use of 
manipulative materials, such as counting — 
blocks, geometric shapes, and so on. 


! -M 


Bliiii __ 


4. 




fill 


f \ f 


— I r f 



0123 0123 0123 



Response Codes: ! 0 = Not well prepared 1=Somewhat prepared 2= Well prepared 

j 3 - Very well prepared 
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Chart D.13 

Teacher Readiness (Part 1) - Science 
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For the following items, please indicate how well prepared you are now to: 



Teach students with physical disabilities. 



Teach classes for students with diverse 

abilities. 



Teach science to students from a variety of 
cultural backgrounds. 



Teach science to students who have limited 
English proficiency. 

Teach students who have a learning 
disability which impacts science learning. 



Encourage participation of females in . 

science. 



Encourage participation of minorities in . 

science. 
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Response Codes: 



0 = Not well prepared 1=Somewhat prepared 2- Well prepared 
3 = Very well prepared 
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Chart D.14 

Teacher Readiness (Part 2) - Science 
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For the following items, please indicate how well prepared you are now to: 



Use/manage cooperative learning groups 

in science. 



Take into account students’ prior 
conceptions about natural phenomena 
when planning curriculum and instruction. 

Provide science instruction that meets 
science standards (district, state, or 
national). 



Integrate science with other subjects. 



Manage a class of students who are using 
hands-on or laboratory equipment. 



Use a variety of assessment strategies 
(including objective and open-ended 

formats). 




illlllii 







1 2 3 



Response Codes: 



| 0 = Not well prepared 1= Somewhat prepared 2 = Well prepared 
I 3 = Very well prepared 
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Chart D.15 



Teacher Opinions (Mathematics): 

a) Beliefs About Student Learning, b) Professional Collegiality 
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Please indicate your opinion about each of the statements below: 



Students leam mathematics best when they 
ask a lot of questions. 

Students master and retain mathematical 
algorithms more efficiently through 
repeated practice than through the use of 
applications and simulations. 

Calculator use should be incorporated only 
after the mastery of basic arithmetic facts. 



All students can learn challenging 
mathematics content. 



Students learn mathematics best in 
classes with students of similar abilities. 



It is important for students to leam basic 
mathematics skills before solving 
problems. 

I am supported by colleagues to try out new 
ideas in teaching mathematics. 



Mathematics teachers in this school 
regularly share ideas and materials. 



Mathematics teachers in this school 
regularly observe each other teaching 
classes. 

Most mathematics teachers in this school 
contribute actively to making decisions 
about the mathematics curriculum. 

I have adequate time during the regular 
school week to work with my peers on 
mathematics curriculum or instruction. 
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Response Codes: 



0 = Strongly disagree 1 = Disagree 2 = Neutral / Undecided I 
3 = Agree 4 = Strongly agree 
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Chart D.16 



Teacher Opinions (Science): 

a) Beliefs About Student Learning, b) Professional Collegiality 
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Please indicate your opinion about each of the statements below: 



Students learn science best in classes with 
students of similar abilities. 

It is important for students to learn basic 
scientific terms and formulas before learning 
underlying concepts and principles. 

Laboratory-based science classes are more 
effective than non -laboratory classes. 

Activity-based science experiences are not 
worth the time and expense for what 
students learn. 






All students can learn challenging science 

content. 






I am supported by colleagues to try out new 
ideas in teaching science. 



— 



Science teachers in this school regularly 
share ideas and materials. 



Science teachers in this school regularly 
observe each other teaching classes. 




Most science teachers in this school 
contribute actively to making decisions 
about the science curriculum. 




I have adequate time during the regular 
school week to work with my peers on 
science curriculum or instruction. 
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Response Codes: 



0 = Strongly disagree 1 = Disagree 2 = Neutral / Undecided 

3 = Agree 4 = Strongly agree 
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Chart D.17 

Grade 4 Mathematics - Content Maps 

Four USI Districts 
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Chart D.18 

Grade 8 Mathematics - Content Maps 

Four USI Districts 
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Chart D.19 

Grade 4 Science - Content Maps 

Four USI Districts 
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Chart D.20 

Grade 8 Science - Content Maps 

Four USI Districts 
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